Papers by Reinald Kim Amplayo
Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing approaches to analyze syntactic knowledge of pre-trained language models have been limited. |
| Approach: | They propose an unsupervised method that extracts constituency trees from PLM attention heads. |
| Outcome: | The proposed method outperforms existing approaches if no development set is present. |
Query Refinement Prompts for Closed-Book Long-Form QA (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can answer questions and produce long-form texts, but the latter is difficult to evaluate since they are subjective in nature. |
| Approach: | They propose query refinement prompts that encourage LLMs to express multifacetedness and generate long-form answers covering multiple facets of the question. |
| Outcome: | The proposed model outperforms fully finetuned models in the closed-book setting and retrieve-then-generate open-book models. |
Scalable and Domain-General Abstractive Proposition Segmentation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Several recent studies have demonstrated the utility of proposition segmentation for downstream tasks. |
| Approach: | They propose a scalable, yet accurate, proposition segmentation model that can be supervised by LLMs. |
| Outcome: | The proposed model improves on training on annotated datasets and shows that it is easy to use. |
Modularized Transfer Learning with Multiple Knowledge Graphs for Zero-shot Commonsense Reasoning (2022.naacl-main)
Copied to clipboard
| Challenge: | Currently, commonsense reasoning systems are limited by expensive data annotations and overfitting to a specific benchmark. |
| Approach: | They propose to transform a commonsense knowledge graph into synthetic QA-form samples for model training. |
| Outcome: | The proposed framework improves performance with multiple commonsense KGs on five commonsensense reasoning benchmarks. |
Retrieval-Augmented Controllable Review Generation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to generate reviews using attribute identifiers are limited and dependent on how well they can capture vector representations of attributes. |
| Approach: | They propose to leverage attributes as inputs for review generation by using reference sets . they propose to use these references to enrich inductive biases of given attributes . |
| Outcome: | The proposed model improves over previous approaches on automatic and human evaluation metrics. |
Rethinking Attribute Representation and Injection for Sentiment Classification (D19-1)
Copied to clipboard
| Challenge: | Existing models that use text attributes to improve sentiment classification use text as a categorical feature. |
| Approach: | They propose to represent attributes as chunk-wise importance weight matrices and consider four locations to inject attributes. |
| Outcome: | The proposed method outperforms the state-of-the-art and outperformed previous models. |
Learning to Plan and Generate Text with Citations (2024.acl-long)
Copied to clipboard
Constanza Fierro, Reinald Kim Amplayo, Fantine Huot, Nicola De Cao, Joshua Maynez, Shashi Narayan, Mirella Lapata
| Challenge: | Large language models (LLMs) are increasingly useful in information-seeking scenarios, ranging from answering simple questions to generating responses to search-like queries. |
| Approach: | They propose to use plan-based models to improve faithfulness, grounding, and controllability of generated content and its organization. |
| Outcome: | The proposed models improve faithfulness, grounding, and controllability of generated content and its organization. |
Informative and Controllable Opinion Summarization (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for opinion summarization use a two-stage extractive and abstractive approach to generate summaries for reviews of a specific target. |
| Approach: | They propose a framework for opinion summarization that condenses all input reviews into multiple dense vectors which serve as input to an abstractive model. |
| Outcome: | The proposed framework produces more informative summaries and allows to take user preferences into account using a zero-shot customization technique. |
Evaluating Research Novelty Detection: Counterfactual Approaches (D19-53)
Copied to clipboard
| Challenge: | Despite its importance, this direction of research has not been explored as much. |
| Approach: | They propose to use counterfactual simulations to evaluate paper novelty detection models . they ask models to differentiate papers at time t and counterf actual paper from future time . |
| Outcome: | The proposed models can be compared against a set of papers with a given date and with different annotations. |
Aspect-Controllable Opinion Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on opinion summarization produces general summaries based on reviews and popularity of opinions expressed in them. |
| Approach: | They propose an approach that generates customized opinion summaries based on aspect queries. |
| Outcome: | The proposed model outperforms the current state of the art and generates personalized summaries by controlling the number of aspects discussed in them. |
𝜇PLAN: Summarizing using a Content Plan as Cross-Lingual Bridge (2024.eacl-long)
Copied to clipboard
Fantine Huot, Joshua Maynez, Chris Alberti, Reinald Kim Amplayo, Priyanka Agrawal, Constanza Fierro, Shashi Narayan, Mirella Lapata
| Challenge: | Recent advances in abstractive summarization have focused on English, but more recently, with the advent of large pre-trained models, the task is becoming more complex. |
| Approach: | They propose an approach to cross-lingual summarization that uses an intermediate planning step as a cross-linguistic bridge. |
| Outcome: | The proposed approach achieves state-of-the-art in terms of informativeness and faithfulness on the XWikis dataset. |
Extractive Opinion Summarization in Quantized Transformer Spaces (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing work on opinion summarization focuses on aggregating opinions among reviews . et al., 2018; see etal., 2019; liu eto, 2019) demonstrate the potential of opinion summaries. |
| Approach: | They propose an unsupervised system for extractive opinion summarization based on vector-quantized variables and an extraction algorithm. |
| Outcome: | The proposed method is validated by human studies showing that judges prefer it over baselines. |
Conditional Generation with a Question-Answering Blueprint (2023.tacl-1)
Copied to clipboard
Shashi Narayan, Joshua Maynez, Reinald Kim Amplayo, Kuzman Ganchev, Annie Louis, Fantine Huot, Anders Sandholm, Dipanjan Das, Mirella Lapata
| Challenge: | Neural generation models often struggle to identify which content units are salient. |
| Approach: | They propose a new conceptualization of text plans as a sequence of question-answer pairs . they propose QA blueprints as QA proxy for content selection and planning . |
| Outcome: | The proposed model improves existing datasets with QA blueprints as proxy for content selection and planning. |
Entity Commonsense Representation for Neural Abstractive Summarization (N18-1)
Copied to clipboard
| Challenge: | Current ELS’s are not sufficiently effective, possibly introducing unresolved ambiguities and irrelevant entities. |
| Approach: | They propose an off-the-shelf entity linking system to extract linked entities and propose Entity2Topic (E2T) module attachable to a sequence-to-sequence model that transforms a list of entities into a vector representation of the topic of the summary. |
| Outcome: | The proposed model improves the performance of the Gigaword and CNN summarization datasets by at least 2 ROUGE points. |
Cold-Start Aware User and Product Attention for Sentiment Classification (P18-1)
Copied to clipboard
| Challenge: | Existing models do not deal with cold-start problem typical in review websites. |
| Approach: | They propose a Hybrid Contextualized Sentiment Classifier that uses word encoder and Cold-Start Aware Attention to pool word vectors. |
| Outcome: | The proposed model performs significantly better on famous datasets despite having less complexity and can be trained much faster. |
Unsupervised Opinion Summarization with Noising and Denoising (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are limited and cannot be easily sourced. |
| Approach: | They propose a supervised learning model which learns to denoise the input and generate original reviews. |
| Outcome: | The proposed model improves on the baselines of abstractive and extractive models on a large dataset with only a few reviews and no ground truth summaries. |
Attribute Injection for Pretrained Language Models: A New Benchmark and an Efficient Method (2022.coling-1)
Copied to clipboard
| Challenge: | Recent models rely on pretrained language models that use metadata as inputs . however, these methods are either nontrivial or cost-ineffective . |
| Approach: | They propose a benchmark for evaluating attribute injection models using eight datasets . they extend adapters to include attributes independently of or jointly with the text . |
| Outcome: | The proposed method outperforms previous methods and achieves state-of-the-art performance on all datasets. |
Text-Blueprint: An Interactive Platform for Plan-based Conditional Generation (2023.eacl-demo)
Copied to clipboard
Fantine Huot, Joshua Maynez, Shashi Narayan, Reinald Kim Amplayo, Kuzman Ganchev, Annie Priyadarshini Louis, Anders Sandholm, Dipanjan Das, Mirella Lapata
| Challenge: | Recent work shows that conditional generation models can be useful to control the text generation process, leading to irrelevant, repetitive, and hallucinated content. |
| Approach: | They propose a web browser-based demonstration for query-focused summarization that uses a sequence of question-answer pairs as a blueprint plan for guiding text generation. |
| Outcome: | The proposed model can be used to generate query-focused summarization text using question-answer pairs as a blueprint plan. |